{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# MNIST using RNN"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 시작하기\n",
    "\n",
    "* 사용할 라이브러리를 import 합니다. \n",
    "* 본 예제는 tensorflow를 사용합니다.\n",
    "* 데이터셋은 tensorflow에서 제공하는 mnist 데이터셋을 사용합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "\n",
    "import os\n",
    "import time\n",
    "\n",
    "tf.reset_default_graph()\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 데이터셋 불러오기\n",
    "\n",
    "* 사용할 데이터셋을 불러온다. \n",
    "* images: 0부터 9까지 필기체로 작성된 흑백영상 (1채널). \n",
    "* label: one_hot 형태로 설정한다. \n",
    "\n",
    "- 0 = [1 0 0 0 0 0 0 0 0 0]\n",
    "- 1 = [0 1 0 0 0 0 0 0 0 0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting MNIST_data/train-images-idx3-ubyte.gz\n",
      "Extracting MNIST_data/train-labels-idx1-ubyte.gz\n",
      "Extracting MNIST_data/t10k-images-idx3-ubyte.gz\n",
      "Extracting MNIST_data/t10k-labels-idx1-ubyte.gz\n",
      "Number of train data is 55000\n",
      "Number of test data is 10000\n"
     ]
    }
   ],
   "source": [
    "mnist = input_data.read_data_sets('MNIST_data', one_hot=True)\n",
    "print(\"Number of train data is %d\" % (mnist.train.num_examples))\n",
    "print(\"Number of test data is %d\" % (mnist.test.num_examples))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 데이터셋 확인하기\n",
    "\n",
    "학습 데이터의 이미지와 레이블을 확인한다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWEAAAFuCAYAAABQqakCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAH8JJREFUeJzt3Xu4JFV57/HvC3IRCKIYQEAQHNBgvAFeQIHxQETQgxoV\nBYwK3hD0KImRY9Qw6EGiRgJiQDxBQIIiFzWKXBQFFBFQ8IYKhJH7OMMwyHUYkOHNH6u2NE3vvav3\n7p61L9/P8+ynd1evrlrVXf3rVatXVUVmIkmqY6XaFZCk2cwQlqSKDGFJqsgQlqSKDGFJqsgQlqSK\nDGFJqsgQlqSKDGFJqsgQniYi4uGI+FztevQSESdGxPUTfO68iHh40HVquewJ13u6iYhzIuILHfd3\narapHScwrxMj4p4B1+/CiPjBIOc5ynJOjYhTh72cfsyqEI6IrSPiWxGxJCLujYhfR8R7Ox7ftNkw\nR/s7rt95NmUeFxGHRMT8iFjW3H4kIlbuKrddU27tAa/3U5r5PmeQ8+2QwESDdDLPnayay15hIuIl\nwM7Ap7oemug5C3ISzx1rnpMSEetFxHERcUtE3B8R10fEf3QV+xfgDRHx7Mkub1AeV7sCK0pEvBz4\nFnAl8HHgXuDpwMYdxRYDb+7x9N2AvYHzJjBPgFOA1wHHA1cALwY+ATwV2L+j3PbAPwMnAHf3uYpj\n2RA4BLge+NUA5zviHUz8C/0TwOEDrEs/JlPv6eSDwPczc8a2+iNiY+ASypfqscCtlO3+hZ3lMvMX\nEfFT4B+At63gavY0K0I4Iv4COAn4dma+YbRymbkU+EqP5+9LCcWz+p1nRGwLvAE4NDMPbSZ/MSKW\nAAdFxOcz86qR4v2tWWt9zTciHp+Z97ctn5nLgeV916o892HgwYk8d7ImU+/pIiL+EtgdeHftugzZ\nFynb0baZeec4ZU8D5kXEAc1nvqrZ0AoA2AdYD/gIQESsERGtgikiNgBeBpyZmZ1h0XaeO1B2tb7W\nNf1Uyuv/xub5hwCfbh67oen+WB4Rm3TV59VNl8eyiLgqInYdp/47AZc3dTixY75vaR6/MCJ+1XSr\n/DAi7gMO61jWWRFxa7O86yLioxGxUtcyHtW32tGt8/cR8c7mecsi4vLmS6nzuY/pEx7p/26zrhEx\nNyJ+1ux+/ndEvKttP/M49T6g6Ta6NyLOi4iNmjIfi4ibI2JpRHwzItbpmucebV6zpuyBzTKWRsSl\nEfHS6NE3GhGrRsShzfoti4ibIuJTEbHqeOsIvIrS2Dq/xevx0oj4WkTc2LGcIyJi9VHKb9a8Nvc2\n6/uxHmUiIj7QvH/3R8TCiPhC9+s2yvyfGhHPaFHuGcArgE9n5p0RsVpEjNXAPB9YC/ib8ea9IsyK\nljClP+xu4KkR8S1gS+C+iDgZOCgzHxjjuXtRWpKnTHCeqzW33S3LkW/gbZrbrzfzeBPwfmBJM31x\nx3N2AP4WOAa4B/g/wBkRsWlm3jFK/X9H6eL4OHAc8KNm+iXNbQJPBs6mfDF8GVjUPPbWZjmfpXS1\n/K9mPn8BHNyxjNH6CPehbOxfaB4/GDgzIjZvWqFjPXfcdY2I5wPnAAuAj1G2548Bt48yz26jLfvN\nwCrA54AnNfU+vQnHnSj9inOaOv0rpVtjxNto8ZpFxHuAo4GLgCOApwHfBP4I3NxRLoBvU7qqjgOu\nBp4NHARs0bxGY9kOWJKZt4xTDsoe2xqU13wJZVf+fcBGNI2FDo8DzgV+AvwjJQQPjYiVM3NeR7kv\nAm8BvgQcBWzWzPN5EfGSju2gl5OBHRm/sbgL5X1cHBHfpzSalkfE94D3ZOaNXeV/Q/k8vgT4r3Hm\nPXyZOeP/gF9QPhD3Av8GvAY4ktJ/dMo4z/0pcAsQE5kn8Npm2t5dz393M/2XHdP+gbJ7vEmPejxM\n2XCe1jHt2c30A8ZZh22acm/p8dgFzTLf0eOx1XpMO5YSMqt0TDsB+H3H/U2b5d0GrN0x/X83y9q9\nY9ohwPKJrCulP/4eYP2OaZtTdkuXd9e9x7qMVu+FwFod0w9rpl8JrNQx/ZSmnp2vxbivGSXgF1MC\nrHN+f9cs5wcd094M/AnYrmue72peyxePs44/BC7vMX2n5vk7jlP3g4GHgI27XrflwL91lf1283o8\nqbn/0mZ93thV7m+a6W/q2g5/0FXuAuChFu/jyOduMfAd4PXA31MaSdcCq/d4ztXAWePNe0X8zZbu\niLWAxwMnZuZBmfnNzPwApWXxpoh4eq8nRcQWlAD7ajbv3ATmeTZwI/CvEfHaiNgkIvYE/h/lw/X4\nPtbje5l5w8idzPw1ZUPbvI959PIAcGL3xOzYQ4iItSJiXeBiSmvpmS3me2pmdv7A+CPKXkWb+o65\nrs3u/c7ANzNzUUe531Nax5NxWmbe23H/sub25Cx92J3TV6W0FEeW3+Y12xZYF/j/XfP7CqUl3On1\nlL2ZayNi3ZE/SkAFpdU3lnV7zLOnrrqv0SznJ5SW6PN7POXfu+5/nrLnt0tz/w3AncD3u+r+c0rj\nZcy6Z+bLMrPN3vpaze2CzHxlZp6RmUcA76Tssezd4zl/pOwBVjdbQnikK6B7fOBXKBvydqM8782U\n3ZzH/FjXdp7Nhr07ZffuDOAGSuAdStkQ7qW9m3tM+yPwxD7m0cutmflQ98SI2CoivhERd1ICcDFl\nFxHgCS3m+6j65iM/mLSp73jruh7lC+y6HuV6TetH97Lvam67d+lHpv95fVq+ZptStqv5nTPLsmt+\nQ9cytgCe1cyn8++aZh7rtViftr9/PLXpJ19C2S4XAxc2y+l+vx8Gft817drmdtPmdg6wDmWPqLPu\ntwFrtqx7G/c3dTy9a/rplFb89j2eEwx+mN2EzJY+4QXAVjzS1znituZ2tFDYC7gmM38+mXlm5u+A\nZ0fEXzXTfwsso+xGXdhuFYDRf8mf7KiKx4yEiIgnUHZl7wQ+SvnALaPsGfwL7b7AJ1PfYa1rG6Mt\ne8w6Deg167YS8GtKH3Cvde/1ZdVpCS2+9Jo9i/MpoXk4JeTvo7TyT6Jd3bvrtxLl87F3j8fg0b93\nTMaC5vZRn8XMfLj5Qum1/k/kkS+NqmZLCF9B2UXaCPjvjukbNreP2Rgi4kWUb/KPDmqeTRiPzH93\nykb6vc4iY63EJExkvnMpG+qrM/PHIxNH67qp4DbKl8ecHo9tsYLrMmIu7V6zGymhNIfyw9xIuZUp\nP9D9sqPsfOA5mXnBBOt0NeP/eAelz30L4O8y888/QkfELqOUX4nSNdS517Flc3tDczuf0mV0SY79\n4/dkXUF5PTfqnBgRq1C6HBZ3TV+ZMka//o9yzJ7uiNMob9Lbu6a/g9Ive2GP5+xNCa+vDnCeQBmH\nSzlIYQGP7s64r7kdd/hOnyYy3+WU9fvzNtIMiTpggPWasKYv9fvAa6IMIwQgIuZQfqmvoe1r9jNK\nC/WdXUPX3sxjW22nARtHxDu7FxYRq0fEGuPU6SfAEyPiaS3qDo/NhA8w+pf4e3vcfxAYGWJ3GqWh\n98/dT4yIlZs9h1G1HaJG+azdBuzTNWxvX8r6fLer/FbA6sCPmQJmRUs4y1EyXwL2bb4dL6L8KPA6\n4JOZubCzfPPB2BO4NEc5yqifeUbE1yiB+1tgbWA/ylCd3TPzvo7ZjnyjfzLK8e1/Ar6VfRw4MYr5\nlF3k/SPiXkooX5qPHbrT6RJKH+yX45FzVoz0kU8V84CXA5dExLGU7flAyu7781ZQHTp3s1u9Zpn5\np4iYRxkCd0FEnEZpAb+N0rLsLH8yZVs8NiJeRgmOlYG/ovzw9XLKqI3RfIcSsLsA3Yfwdtb9asp2\n8tkoR5/dTdmWR/vifgB4RUScBFxK+d1jN+CwzFzSrOcPoxzq/38j4nmUMPwTpcX8esoQv6+PUfdW\nQ9Qy88GI+CCl2+RHzTDRTZv5/xD4RtdTXk75DIw7dnqFqD08Y0X9UTbcj/FIP901wPtGKftyyoY7\n3tCvVvOkHDb6G8obfztlw3v2KPP8J+Amysb65+Fqzf9H9Sj/e+D4Fuv/Kko4PdDM6y3N9AvoGCbX\n9ZwXUz7091L6Hj9J+TB3D206AZjfcX/TpsxBPea5HPhYx/1D6BqG1M+6UroAfkbpmriW0vr5DHBf\ni9ekVb15ZDjX33ZNf2szfet+X7Om7IHNOi2ltFhfTBkS+Z0e29kHKYecL222ocspBwqt1WI9vwl8\nd5R16nwfn0E5NP8uSv/qscBfd24vHa/bXZQvjnMpw+8WdL6vXct6e1PfeymNgV80r0vn0MILKIdW\n0zVt3CFqHeX3pHwhLW3qcxSwZo9yP6GMaqqeS5lZxr5KM0lEfAPYKjPb7MpOGc2BGYspR2cO7DDj\niHgpJdCemZnzxys/kzUt8p8Bz88y7LG62dInrBkqIlbrur8FZdd4oj9krRDR+5Djt1KO0Bto3TPz\nYkpXwIcGOd9p6mDg9KkSwIAtYU1vEXErpS/w95Td4/0pR6RtPZVbfVHO6XEEZez4Esowtv0o3Vbb\nZo9x25qZZsUPc5rRzqWcb2MDSn/3JcA/TeUAbtxA6TN+H6X1ewflIJ4PG8Czy5Tsjohydqnrm7Mu\nXRoRL6hdp0GIcmL17hPF/7Z2vSYiInaIcjL7W5v12KNHmY9HxIIoZwn7XjN8bKAy8+2ZuXlmrpGZ\nT8xy2Oovx3/m2MZbv4g4ocd7eXYf9b4xM1+TmRtm5urN7Tsz8/bJ1n0sEfHhKGezuzsiFjVH923Z\nVWa1iPj3iLg9Iu6JiDMiYlBHtw1Vy/W7sOt9Wx4Rx9Sq85QL4Yh4I+UMVIdQjlf/JXBeREyJ47wH\n4CpgfUrLbQPKSU6mozUpv3IfSI9haxFxMGXc6LspZ+O6j/I+tjn94lQw5vo1zuHR7+VeK6Zqk7ID\n5extL6KM2lgF+G4zdn3EkcArKUPUdqQcgHTmCq7nRLVZv6Sc3W3kvXsKNfvLaw/P6DF85FI6hidR\nxjLeAnyodt0GsG6HAFfWrscQ1uthYI+uaQvoGOpFGR99P7Bn7foOaP1OAL5eu24DWLcnN+v30o73\n6QHgtR1lntGUeWHt+k52/ZppFwBH1K7byN+Uagk3Bz1sQzkSCoAsr9r5jH6Snelmi2YXd35E/GdE\nPLV2hQYtIjajtDA638e7KWcdmynvI8DcZpf36og4JiKeVLtCE7AOpWU4cj7qbSi/FXW+d9dQxq5P\nx/eue/1G7BMRi6NcNOCTXS3lFWqq/TD3ZMrA9O6T4iyifBtPd5dSjoq6hrILNA/4YUT8dT76yLnp\nbgPKht/rfdzgscWnpXMou+jXU64reDhwdkRs1zQcprxmXPKRwMWZOfLbxAbAg/noU5DCNHzvRlk/\nKOeBvpGyt/YcyhVtRo7iW+GmWgiPZsqcdm4yMrPzQqFXRcTllI1hT8ru7Uw3I95HgMw8rePubyLi\n15TDfucyxccodziGch6FNr9LTMf3bmT9XtI5MTM7D9/+TUQsBM6PiM2ywsVQp1R3BOVwzOWUDvNO\n6/HYVtW0l5l3UQ61HfiogcoWUj60s+J9BGg+vLczTd7LiPg85aCWuZm5oOOhhcCqEbF211Om1XvX\ntX5/GKf4ZTxyVrsVbkqFcGb+iXISm51HpjW7FDvzyDXRZoyIWIuyKzveRjKtNIG0kEe/j2tTfrGe\nce8j/PmS6+syDd7LJqBeDbwsM2/qevgKyonQO9+7LYFNKOdcmPLGWb9enk9p5Vd576Zid8QRwEkR\ncQXlpB8HUS4Nc2LNSg1CRHyGch2uGynnPj2UssGPdrrMKSsi1qS0HEbOxLV5RDwXuCMzb6b0xX00\nIq6jHJjwCcoolylxDtfxjLV+zd8hlD7hhU25T1H2as577NymjmY87F7AHpQL047srdyVmcsy8+6I\nOB44IiL+SDk5z+eAH2fm5XVq3d546xcRm1NOU3s25UjF51Iy56LMvKpGnasPzxhlWMkBlA/u/ZRv\n321r12lA6/VVShDdT/m1+SvAZrXrNcF12Yky9Gd519+XOsrMo/z4sZQSTnNq13sQ60c5F+25lABe\nRjlk+ljgL2vXu8V69Vqn7rOkrUYZa3s7JYRPB9arXfdBrB+wMeX8w4ub7fIayo+q456Nblh/njtC\nkiqaUn3CkjTbGMKSVJEhLEkVGcKSVJEhLEkVGcKSVFH1gzUiYl1gV8q44GV1ayNJA7E65XJb52Xm\nkrEKDi2EI+JAymW6N6CcmP19mfnTHkV3pZzVSJJmmn0oB2WNaijdEX1eHeOGYdRBkqaAG8YrMKw+\n4YOA4zLzy5l5NeUKuEspV5PtZheEpJlq3HwbeAjPkqtjSNJADKMlPNbVMabVmfkladhW5BC16Xhm\nfkkaqmGE8Ky6OoYkTcbAQzhn2dUxJGkyhjVOeMZeHUOSBmkoIZyZpzVjgj9O6Zb4BbBrZi4exvIk\nabqqfmWNiNia0n0hSTPNNpl55VgFPIGPJFVkCEtSRYawJFVkCEtSRYawJFVkCEtSRYawJFVkCEtS\nRYawJFVkCEtSRdWvtizNRjvuuGPrshdeeGHrsiutZLtquvEdk6SKDGFJqsgQlqSKDGFJqsgQlqSK\nDGFJqsgQlqSKDGFJqsgQlqSKDGFJqsjDlqUBmTNnTuuy8+bNa1229hXRNVy2hCWpIkNYkioyhCWp\nIkNYkioyhCWpIkNYkioyhCWpIkNYkioyhCWpIkNYkirysGVpQHbbbbfWZXfaaafWZY8//viJVEfT\nhC1hSarIEJakigxhSarIEJakigxhSarIEJakigxhSarIEJakigxhSarIEJakigxhSaooBn057Yg4\nBDika/LVmbnVKOW3Bq4YaCWkAdlwww1bl73gggtal1177bVbl912221bl7311ltbl9UKsU1mXjlW\ngWGdwOcqYGcgmvsPDWk5kjStDSuEH8rMxUOatyTNGMPqE94iIm6NiPkR8Z8R8dQhLUeSprVhhPCl\nwNuAXYH9gc2AH0bEmkNYliRNawPvjsjM8zruXhURlwM3AnsCJwx6eZI0nQ19iFpm3gVcC8wZ9rIk\naboZeghHxFrA04E/DHtZkjTdDDyEI+IzEbFjRGwaEdsD36AMUfvqoJclSdPdMIaobQx8BVgXWAxc\nDLw4M5cMYVmSNK0N44e5vQY9T0maqbzkvWaV9ddfv6/y73nPe1qXnTOn/W/P+++/f+uyHoo8s3kC\nH0mqyBCWpIoMYUmqyBCWpIoMYUmqyBCWpIoMYUmqyBCWpIoMYUmqyBCWpIo8bFmzymGHHdZX+f32\n26912Z///Oety5588sl91UMzly1hSarIEJakigxhSarIEJakigxhSarIEJakigxhSarIEJakigxh\nSarIEJakijxsWbPK6173ur7KZ2brsmeddVbrssuWLeurHpq5bAlLUkWGsCRVZAhLUkWGsCRVZAhL\nUkWGsCRVZAhLUkWGsCRVZAhLUkWGsCRV5GHLmvYOP/zw1mWf8IQn9DXvM888s3XZY489tq95S2BL\nWJKqMoQlqSJDWJIqMoQlqSJDWJIqMoQlqSJDWJIqMoQlqSJDWJIqMoQlqSIPW9aUNGfOnNZl99ln\nn6HV49RTT21dduHChUOrh2auvlvCEbFDRHwrIm6NiIcjYo8eZT4eEQsiYmlEfC8i2n+iJGkWmUh3\nxJrAL4ADgex+MCIOBt4LvBt4IXAfcF5ErDqJekrSjNR3d0RmngucCxAR0aPI+4FPZOa3mzJvARYB\nrwFOm3hVJWnmGegPcxGxGbAB8P2RaZl5N3AZsN0glyVJM8GgR0dsQOmiWNQ1fVHzmCSpw4oaohb0\n6D+WpNlu0CG8kBK463dNX4/Hto4ladYbaAhn5vWUIN55ZFpErA28CLhkkMuSpJmg79EREbEmMIfS\n4gXYPCKeC9yRmTcDRwIfjYjrgBuATwC3AP81kBpL0gwykSPmtgUuoPTxJvDZZvpJwH6Z+emIWAM4\nDlgH+BGwW2Y+OID6StKMMpFxwhcxTjdGZs4D5k2sShK84x3vaF12o402al22n6snA5x99tl9lZf6\n5Ql8JKkiQ1iSKjKEJakiQ1iSKjKEJakiQ1iSKjKEJakiQ1iSKjKEJakiQ1iSKvJqy5qUVVZZpXXZ\nY489tnXZ/fbbbyLVGddJJ53UV/n7779/KPWQRtgSlqSKDGFJqsgQlqSKDGFJqsgQlqSKDGFJqsgQ\nlqSKDGFJqsgQlqSKDGFJqsjDljUpu+yyS+uy++67b+uymdm67E9/+tPWZc8666zWZQHWWGON1mWf\n+cxnti7bz+vWzxWi58+f37qspgZbwpJUkSEsSRUZwpJUkSEsSRUZwpJUkSEsSRUZwpJUkSEsSRUZ\nwpJUkSEsSRUZwpJUUfRzjP5QKhCxNXBF1UroUfo5B8Jll13Wuuxaa601keqMa4cddmhddunSpX3N\n++ijj25ddvvtt+9r3m3ddNNNrcvOnTu3ddkbb7xxArVRn7bJzCvHKmBLWJIqMoQlqSJDWJIqMoQl\nqSJDWJIqMoQlqSJDWJIqMoQlqSJDWJIqMoQlqaK+L3kfETsA/whsAzwFeE1mfqvj8ROAt3Y97dzM\n3H0yFdWK85GPfKR12WEditzPIbXrr79+67KnnHJKX/VYbbXV+io/DJtssknrsscff3zrsrvssstE\nqqMBm0hLeE3gF8CBwGgnnjgHWB/YoPnba0K1k6QZru+WcGaeC5wLEBExSrEHMnPxZComSbPBsPqE\n50bEooi4OiKOiYgnDWk5kjSt9d0SbuEc4EzgeuDpwOHA2RGxXdY+b6YkTTEDD+HMPK3j7m8i4tfA\nfGAucMGglydJ09nQh6hl5vXA7cCcYS9LkqaboYdwRGwMrAv8YdjLkqTpZiLjhNektGpHRkZsHhHP\nBe5o/g6h9AkvbMp9CrgWOG8QFZakmWQifcLbUvp2s/n7bDP9JOAA4DnAW4B1gAWU8P3nzPzTpGsr\nSTPMRMYJX8TY3RivmHh1JGl2GcYQNU1Br3rVq1qX3WeffVqXHdaow0033bR12TPOOKN12dGPL+pt\nuo2q9ArK048n8JGkigxhSarIEJakigxhSarIEJakigxhSarIEJakigxhSarIEJakigxhSarIw5Zn\niX6uSNzvob3TyXRct0WLFrUu+/a3v32INdEw2BKWpIoMYUmqyBCWpIoMYUmqyBCWpIoMYUmqyBCW\npIoMYUmqyBCWpIoMYUmqyMOWZ4lnPetZrcv2c4Xh6XY14n4Na/2WLFnSuuyuu+46lDpoarAlLEkV\nGcKSVJEhLEkVGcKSVJEhLEkVGcKSVJEhLEkVGcKSVJEhLEkVGcKSVJGHLc8S/VxtWROzYMGC1mXf\n9a53tS77q1/9aiLV0TRhS1iSKjKEJakiQ1iSKjKEJakiQ1iSKjKEJakiQ1iSKjKEJakiQ1iSKjKE\nJamivg5bjogPA68FngncD1wCHJyZ13aUWQ04AngjsBpwHnBAZt42qEqrf+eff37rsnvttdcQazJ4\nd955Z+uyZ555Zl/z7qf8JZdc0rrsPffc01c9NHP12xLeATgaeBGwC7AK8N2IeHxHmSOBVwKvA3YE\nNgT62/IlaZboqyWcmbt33o+ItwG3AdsAF0fE2sB+wJsy86KmzL7A7yLihZl5+UBqLUkzxGT7hNcB\nErijub8NJdi/P1IgM68BbgK2m+SyJGnGmXAIR0RQuh4uzszfNpM3AB7MzLu7ii9qHpMkdZjM+YSP\nAbYCXtqibFBazJKkDhNqCUfE54HdgbmZ2Xkm64XAqk3fcKf1KK1hSVKHvkO4CeBXAy/LzJu6Hr4C\neAjYuaP8lsAmwE8mUU9JmpH6HSd8DLAXsAdwX0SMXDPnrsxclpl3R8TxwBER8UfgHuBzwI8dGSFJ\nj9Vvn/D+lL7dC7um7wt8ufn/IGA5cAblYI1zgQMnXkVJmrn6HSc8bvdFZj4AvK/5kySNITLrDlqI\niK0pfckaon6utrz99tu3LnvwwQe3LvuCF7ygddljjjmmddmjjjqqddnrrruudVlpALbJzCvHKuAJ\nfCSpIkNYkioyhCWpIkNYkioyhCWpIkNYkioyhCWpIkNYkioyhCWpIkNYkirysGVJGh4PW5akqcwQ\nlqSKDGFJqsgQlqSKDGFJqsgQlqSKDGFJqsgQlqSKDGFJqsgQlqSKDGFJqsgQlqSKDGFJqsgQlqSK\nDGFJqsgQlqSKDGFJqsgQlqSKDGFJqsgQlqSKDGFJqsgQlqSKDGFJqsgQlqSKDGFJqsgQlqSKDGFJ\nqsgQlqSKDGFJqsgQlqSKDGFJqsgQlqSKDGFJqqivEI6ID0fE5RFxd0QsiohvRMSWXWUujIiHO/6W\nR8Qxg622JM0M/baEdwCOBl4E7AKsAnw3Ih7fUSaBLwLrAxsATwE+NPmqStLM87h+Cmfm7p33I+Jt\nwG3ANsDFHQ8tzczFk66dJM1wk+0TXofS8r2ja/o+EbE4In4dEZ/sailLkhp9tYQ7RUQARwIXZ+Zv\nOx46BbgRWAA8B/g0sCXw+knUU5JmpAmHMHAMsBXwks6JmfkfHXd/ExELgfMjYrPMvH4Sy5OkGWdC\n3RER8Xlgd2BuZv5hnOKXAQHMmciyJGkm67sl3ATwq4GdMvOmFk95PqXfeLywlqRZp68Qbsb77gXs\nAdwXEes3D92VmcsiYnNgb+BsYAnwXOAI4KLMvGpw1ZakmaHflvD+lFbthV3T9wW+DDxIGT/8fmBN\n4GbgdOCwSdVSkmaofscJj9mHnJm3AHMnUyFJmk08d4QkVWQIS1JFhrAkVWQIS1JFhrAkVWQIS1JF\nhrAkVWQIS1JFhrAkVWQIS1JFhrAkVWQIS1JFhrAkVWQIS1JFhrAkVWQIS1JFhrAkVWQIS1JFhrAk\nVWQIS1JFhrAkVTQVQnj12hWQpCEZN9+mQgg/rXYFJGlInjZegcjMFVCPMSoQsS6wK3ADsKxqZSRp\nMFanBPB5mblkrILVQ1iSZrOp0B0hSbOWISxJFRnCklSRISxJFRnCklSRISxJFRnCklTR/wBdomPG\nLMhzKAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fe2882d0810>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWEAAAFuCAYAAABQqakCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAH6ZJREFUeJzt3XuYJFV9//H3lxVBFwGBIEaRi4hGiShrdEkEN+IvBFA0\nasRbBMnziwa8RJO4AaOLGgniD0TRNSqIGgL5gTfUCKgoRDCAAiqLoki4yGW5GsDlIux+88eplt6m\nZ6Z6pnvPTM/79Tz97Hb1qapT3T2fPn3q1OnITCRJdaxXuwKSNJ8ZwpJUkSEsSRUZwpJUkSEsSRUZ\nwpJUkSEsSRUZwpJUkSEsSRUZwnNUROwfEWsiYpfademnqdu7p7nu1RHx6WHXqeW+p13vuSQito6I\neyJi165ln4mIq6a5vTUR8ZEh1m+bZpuvG9Y2J9jPZhHx64jYc5T7mcxYhXBELIyI90TE6RFxW9sX\nMSIWRMRPmvJv73lsWbN8otuuPeXf1Gzr3oi4LiKOiohHTnebEfHXEbH/BFWf9jXnEbFXRCyb7vot\nJNOv35oZrDtTM6n3XPJu4PzM/K+uZWNz7F2NlN7b6ojYslMuM28HjgP+qVZdH1ZrxyOyBfAu4Brg\nh8CSluu9Fdia/m/ALwBX9Fn+z8BGwPc7CyLiA8DfA6cAxwBPBd7c/LvXdLYJHATcAny23aG0tnez\n7fcMebsdjwAemOa6T6YEcQ0zqfecEBFbAK8D/qJ2XUYsKXlwdc/y/+m5/y/AWyJiSWaevQ7qtZZx\nC+EbgK0y8+aIWMTaYdZX86n4LuAI4H29j2fmCmBFzzqPBx4PfDIzH2iWbQW8DfhsZr6+q+wVwEci\nYp/M/I9Btjli0bpgxAJgvcy8v+06mfmbadWqrNt6P8M2k3rPIX8B3A98rXZF1oEzMvPiyQpk5uUR\nsQI4ADh7XVSq21h1R2Tm/Zl584CrHQH8FPi3AdZ5dfNv9zq7AguA/99T9t8pgffKlts8qbOg6Z97\nGrCk6+vUt3vW2yAijo6Im5u+rS9GxOaT7SgiTqC0gun+mtbc7/TFvT0i3hoRvwDuBX4vItaPiPdG\nxA8i4n+a/f1nRCzps4+1+lYj4rBm2RObvsdfNdv4dERs2LPuWn3CXV8t/3CqY43isIi4PiJWRcRZ\nEfF7vduc5LmZqN5PiogTmzrfHBHvbR7fOiK+HBF3RMSNfbqzBnnONouIf2229auIOCEinh59utUi\n4skR8fko3W73RMT3I+JFUx1f48WUroi7WzwffxcR50XErRFxd3McL5uk/Ksj4vKmTj+IiN36lPnd\n5nVfGaXbbkVEHNiiLg9rjnurKY9w7fU2ioipsu5bQNvnb6jGKoQHFRHPpnwt+xsG6wt7NXBdZp7b\ntWyD5t97esp23uiLWm7zu13L3gpcR/mQeA3wWuD9XY8H8FHg94HDgOWUN9JHp9jXvwDfbP7f2W7v\nV9MDgTcBnwD+Frgd2LhZ/h3gHcAyShfQGRHx9Cn22Xl+TwEWAv9A+cDav9lOv7K9jmXqYz2C0t95\nIfB3lG6fMyndDNPRqUvnw3UpcD7wzoj4G+AblNdoabOvD0bEc7vWb/WcRURQWqb7AScAhwKPpXRD\nrfV8RMTTmjo8mdKF9Xbg18CXI+LFkx1M863mD4BLWh7/W4CLKd8WD6G0oE+JiL36lF0CfAj416b8\nZsDpEfHUrv1vCVwAPB/4SLP9K4DjIuItU9TlcZS/hcNb1j0oLds7gbsj4rSI2GGCsj8ANu2u6zqT\nmWN5o4TeGuB1k5S5APjX5v/bNOXfPsV2n9qUO7xn+TOb5Yf2LN+zWX7HoNtsHrsU+Haf5fs365zR\ns/wo4DfAo6Y4jmOB1X2Wd56HXwGb9TwWwMN6lm0M3Ah8qmf5GuDdXfeXNcs+2VPuC8DNPcuuAj49\n6LECWzb3P99T7t3N+p/uPd4+xz9RvZd3LVsPuJbSd/y3Xcs3AVb11L3Vcwa8tNnPm3rKfgtY3f0+\nbpZd0me75wKXT3F82zf7OajPYycA/92zbIOe+wuAHwPf7PO8rQae0bVsa0oj5PNdy46jfGht2rP+\nSZQP+g163ofdx71Ns4/jW7yOfw4cT2lg7Es59/Fr4CbgcX3KL2729/Kptj3s27xtCUfE6ylf9ZcO\nuOprKS2Tk7oXZuYllFBfGhEHNF/r96K0Ou9n8pZY3222kMAne5Z9l/KHss2A2+r1+Sxnjh/cWdHp\nA4+IeDTwcEoros1QuaS0rHvru3lEbNRi3amOdY/m/sd7yh3bom5T7fv4397JXEM55qAEV2f5HcDP\nKEHXWdb2OduT8gFyXM++P0ZX/32z/h8DpwKbRMTmnRulVf6kiHjsJMfS6b75VYvjJjPv69r3psCj\nKc97v9f7e5n5w651fwmcBvxJ09KH8mHzVWBBn7pvMsF2O9u7JjMXZOZftqj3qZn5l5l5YmZ+JTOX\nUZ7jLYB39lml83xsMdW2h23cTsy1EhGPonylOTIzbxhw9VcCl2U5udbrpZSvrcdT/nAeAI6mfE3b\ncZrbnMove+533kyPnsa2ul3db2GU4XJvB54CrN/10H+33O61Pfe76/vrKdad6lg7YfyL7kKZ+auI\naBU6k+it9x3Avb0fVM3yzboXtHzOtgFuzMx7e7b3i577O1DeW++j/7CqpHwjuLH/YTxYrSkeL4Ui\nXkgJrWfwYJcb9B+90ltXgJ8DrwC2aHJ4U+CvgDf0Kdup+0hk5nkRcQHwgj4Pd56PdT5Eb16GMGUY\n2fqUvq3OH+7Wzb+PbpbdkD1n6Zu+vm2ZoPWcmTcCu0fEE4GtgCuyjNS4nvJmfIipttnC6gmWtx79\nMIHevm0i4rWUlt8XgSOBm5v9H0pX628KM6lvv3Wj5boz1W/fUx7LkJ6zbp1vr/+P0tfdT78w7Lit\n+XfKD+nmpNpplH7Vv6YE+/2UPu5XtagrrP3adOp+IhMPufxxy+1O1y/p3yDqPB+3jnj/DzFfQ3hr\nypP+k57lSfnUP5TSx9v7hngNpQXw75NtPDOvBK4EaDr6HwtMdGZ+qm2O6pN5Ott9GXBlZr68e2Fn\npEAl3cdxTfPvDl3/JyI2Y+bfDKar7XN2DWUUzIY9reEn9ZTrtJ7vz8zekTJtXEv5gN2uRdmXNmX3\nzK5hkxExUXdAb12hBN7dlHAL4C5gwTTrPgzbU8bd99qO8l766bqtzvwdHfFh4M+Al3Td/ooH+/he\nQjk59FsR8TDg5cB3m76uKTX9YEdSTtb09oW23eYqyle4YVvV1GHjAdZZzUPP1D+HMjxvNjiLUseD\nepa/uUJdOto+Z2dS+or/b1e5AA7uXj8zb6G0TN/Qb6hWlAsxJtSE6Q+AZw1Q99821iJiW8oQt352\nja7L6CNia8pJsTObvvE1lBOxL2tGeAxU90GGqPXbVkTsTTlhf3qfVRZRTp73NsxGbuxawhFxMCW0\nHtcs2rd5MwB8JDPvak4e/LBnvU63xGWZ+dU+m/5TykmNCccTR8QxwIbNttentHKfBeyfmddNZ5vA\nRcAbI+KdlK+ZN2fmdzq7nKgqk2yve7sBHBsRZ1JGSvSOce71NeClEfFl4D8orYo3AJdRrvQbpSmP\nten6+TDw9og4DTgD2JnyPN9CnUty2z5nX6YMqzsqIp4EXE4JsM4HcHfdD6acHLs0Ij5FaR0/hhLs\nj6N8i5vMacA/RcRGmTlZP/zXKH3ZZ0bESc0+DqIMKes3JHEFZUjasZSTjH/d1PuwrjL/QDlHckFT\n959Q+tAXUYatTRbEnSFqn6F0iUzmexFxCeUD545m+6+nfOP45z7l/w/lhOG6t66HY4z6RmnBrp7g\n9oRJ1usMf+k7RI0ycuEeeobW9JTZnzKm8k7KpZHfAHafpHybbW4JfKXZ3mqa4WrNvlYDu/SUf16z\nfML9NuXWo1xavZJyAnF1z/PwtgnWW0r5o7+b8gbfi/Lt4cqecquBd3XdX9Ys6x321jmOJ3Qt+2+6\nhiENcqyUUD4MuJ5you8blK/EtwAfa/H+aVvvE+gz7JAyHvhH03zONqOMsf0fynCt4yjBugb4856y\n2zbbuJ5yMc21lHB9SYtj/B3gPuDVfY6pt04HUD4Q7qZ8cLyu85z0ed4+TOkr/llT/vvAbn32vwVl\njPDVTd2vb16nA/v8PU53iNp7KQ2N25t9XEUZJfM7fco+pXmOl0y13VHcoqmENLYiYhPKSIp3Zma/\nVtCsFREvoXyFf26uPdnOTLd7HLBjZu4+rG3OVc032OdmZpsumqGbr33CGlPRcwl0422Ur8Vnr9va\nDCYiNui5vx6lP/tOyjesYXoP8KzomQVwvmlO2h5I/7HD68TY9Qlr3tsvIg6g9L+uAnajjMM+Y5gt\nyRE5Nsq0p/9FGZP7MsqVXIdk10UTw5DlRPAjpyw45rKM8x7k5PTQGcIaNz+mjGV9B+WP6ybKfAbv\nqlmplr5DORG2D+UE7y8olzH3XgGoMTIruyMi4uCIuCrKTEznR8Qf1K7TMET/ydzX+ZCYYYiI3SLi\nK1FmK1sTEfv2KfPeiLghyuxb35xk8pShycxLMvNPMnPLzNwwM7fJzL/NFjOGdZvq+KLMcNb7Wn59\nhnU/OTP/IDMfnZmPyMzfH3YAR8QhEXFhRNwZETdFxJciYseeMhtExMeizJx2V5TZ2kZ2JdswtTy+\ns+OhE70vr1XnWRfCEbEfZWKWZZShNj+iDJFZ59d0j8gKylCfrZrbcycvPmstpAzFW2sca0dELKXM\nwvYG4NmUroEzI+Lh67KSMzDp8TVOZ+3Xsu1VZDXtRhkl8BzK5bvrA9+IiO65TY6htMZfBuwO/C7l\n5OBc0Ob4OvOQdF67x1K+OdVRY0jGFENLzgc+3HU/KLMuvaN23YZwbMuAi2vXYwTHtQbYt2fZDXQN\nc6N0DdwDvKJ2fYd0fCcAX6xdtyEc2xbN8T2363W6D/izrjKdXzp5du36zvT4mmXfAY6uXbfObVa1\nhCNifcqg6rM6y7I8a99i9lyVNVNPar7iXhllkvCtp15lbomI7SgtjO7X8U7KLHPj8jpCucz4piiT\nmC9vzrTPNZtSWoadiYgWUc4Vdb92P6OMQ56Lr13v8XW8JiJuiYhLI+LwnpbyOjXbTsxtQZmK8Kae\n5TdRPo3nuvMpg99/RvkKdBjwnxGxU2auqlivYduK8sbv9zoO9KsIs9jplK/oVwFPpFyF9fWI2LVp\nOMx6zWXRxwDn5oOX624F/Kb50Ow25167CY4PyhWq11C+rT2dMrXAjpQpBNa52RbCEwnG4FdgM7N7\n1qsVEXEh5c3wCrrmpR1jY/E6AmTmKV13L4uISymTNi2hfN2dC5ZTflCgzXmJufjadY7vj7oXZmb3\nnM2XRcRK4FsRsV1mrjVnzLowq7ojKDMtraZ0mHfbkoe2qua8LJOA/5wy69c4WUn5o50XryNA88d7\nK3PktYyIj1J+cXtJrj2n9krg4fHQiZ3m1GvXc3xTza18AeX9WuW1m1UhnGX+3osov5AA/PYrxR7A\n92rVa1Si/JrEE5l6Au45pQmklaz9Om5MOWM9dq8j/PbXsjdnDryWTUC9GPjjzOydrP4iylwi3a/d\njsATKBeRzHpTHF8/z6S08qu8drOxO+Jo4LMRcRFlVqm3Ua7s+UzNSg1DRHyQMlPTNZQZod5DecOf\nXLNe0xERC3nwVx4Ato+InYHbs1yNdQzwj1F+rflqyi9BXEeZZGbWm+z4mtsySp/wyqbcByjfaiaa\naH1WaMbDvooyQ9uqiOh8W7kjM+/NzDsj4njg6Ci/RnIXZbKd8zLzwjq1bm+q44uI7Sk/qvt1ygT3\nO1My55yc3i/bzFzt4RkTDCs5iPKHew/l0/dZtes0pOM6mRJE91DONp8EbFe7XtM8lufx4I87dt+6\nf+TyMMrJj7sp4bRD7XoP4/goV7OdQQngeykzpH2cPjN0zbbbBMfUO1vZBpSxtrdSQvhUYMvadR/G\n8QGPp8whckvzvvwZ5aTqRrXq7CxqklTRrOoTlqT5xhCWpIoMYUmqyBCWpIoMYUmqyBCWpIqqX6wR\nEZsDe/LgL69K0ly3IeUXsc/MzNsmKziyEI6Ig4G/o8y89CPgzZn5/T5F96TMaiRJ4+Y1lIuyJjSS\n7ogBfx3j6lHUQZJmgaunKjCqPuG3AZ/IzM9l5uXAGymXCB7Yp6xdEJLG1ZT5NvQQnie/jiFJQzGK\nlvBkv44xp2bml6RRW5dD1ObizPySNFKjCOF59esYkjQTQw/hnGe/jiFJMzGqccJj++sYkjRMIwnh\nzDylGRP8Xkq3xA+BPTPzllHsT5Lmquq/rBERu1C6LyRp3CzKzIsnK+AEPpJUkSEsSRUZwpJUkSEs\nSRUZwpJUkSEsSRUZwpJUkSEsSRUZwpJUkSEsSRUZwpJUkSEsSRUZwpJUkSEsSRUZwpJUkSEsSRUZ\nwpJUkSEsSRUZwpJUkSEsSRUZwpJUkSEsSRUZwpJUkSEsSRUZwpJUkSEsSRUZwpJUkSEsSRUZwpJU\nkSEsSRUZwpJUkSEsSRUZwpJUkSEsSRUZwpJUkSEsSRUZwpJUkSEsSRUZwpJUkSEsSRUZwpJUkSEs\nSRUZwpJUkSEsSRU9bNgbjIhlwLKexZdn5lOHvS+Nr0WLFrUuu8cee4ywJqOxYMGC1mXf//73j6QO\nEdG67NKlS1uXPfLII6dTnXlr6CHcWAHsAXRe5QdGtB9JmtNGFcIPZOYtI9q2JI2NUfUJPykiro+I\nKyPixIjYekT7kaQ5bRQhfD5wALAn8EZgO+A/I2LhCPYlSXPa0LsjMvPMrrsrIuJC4BrgFcAJw96f\nJM1lIx+ilpl3AD8Hdhj1viRprhl5CEfERsATgRtHvS9JmmuGHsIR8cGI2D0itomIPwS+RBmidvKw\n9yVJc90ohqg9HjgJ2By4BTgXWJyZt41gX5I0p43ixNyrhr1NSRpXo7pYQ3PYxhtv3LrsiSee2Lrs\n4sWLW5fdcMMNW5dduHC8Rz9mZvXt7rXXXq3LetnyYJzAR5IqMoQlqSJDWJIqMoQlqSJDWJIqMoQl\nqSJDWJIqMoQlqSJDWJIqMoQlqSIvW9ZDHHHEEa3L7rPPPiOsiWaLn/70p7WrMLZsCUtSRYawJFVk\nCEtSRYawJFVkCEtSRYawJFVkCEtSRYawJFVkCEtSRYawJFXkZcvzxEYbbdS67JOf/OQR1qSdlStX\nti57zTXXjLAm7Z188smty15yySUjrMnw/ehHP6pdhbFlS1iSKjKEJakiQ1iSKjKEJakiQ1iSKjKE\nJakiQ1iSKjKEJakiQ1iSKjKEJakiL1ueJ3baaafWZZcsWTK6irT0ohe9qHXZiy++eIQ1kUbLlrAk\nVWQIS1JFhrAkVWQIS1JFhrAkVWQIS1JFhrAkVWQIS1JFhrAkVWQIS1JFXrY8T2ywwQa1q8B9993X\nuuyKFStGWBNp9hi4JRwRu0XEVyLi+ohYExH79inz3oi4ISLujohvRsQOw6muJI2X6XRHLAR+CBwM\nZO+DEbEUeBPwBuDZwCrgzIh4+AzqKUljaeDuiMw8AzgDICKiT5G3Au/LzK82ZV4H3AS8BDhl+lWV\npPEz1BNzEbEdsBVwVmdZZt4JXADsOsx9SdI4GPboiK0oXRQ39Sy/qXlMktRlXQ1RC/r0H0vSfDfs\nEF5JCdzH9Czfkoe2jiVp3htqCGfmVZQg3qOzLCI2Bp4DfG+Y+5KkcTDw6IiIWAjsQGnxAmwfETsD\nt2fmL4FjgH+MiF8AVwPvA64DThtKjSVpjEznirlnAd+h9PEmcFSz/LPAgZl5ZEQ8EvgEsCnwXWCv\nzPzNEOorSWMlMuueL4uIXYCLqlZiHjj77LNbl91tt91GUodzzjmnddlzzz23ddlLL720ddlTTz21\ndVlpCBZl5qQ/B+4EPpJUkSEsSRUZwpJUkSEsSRUZwpJUkSEsSRUZwpJUkSEsSRUZwpJUkSEsSRV5\n2fIcts8++7Que9pp7edP6v+rVbPXmjVrWpddvXr1QNu+/vrrW5c9/PDDW5c9/vjjB6qH5iwvW5ak\n2cwQlqSKDGFJqsgQlqSKDGFJqsgQlqSKDGFJqsgQlqSKDGFJqsgQlqSKpvOT95olBrm8eK5dijyI\n9dZr35YYpCzAtttu27rsxz/+8dZld95559Zl3/KWt7Quq7nHlrAkVWQIS1JFhrAkVWQIS1JFhrAk\nVWQIS1JFhrAkVWQIS1JFhrAkVWQIS1JFhrAkVeTcEZqVzjrrrNZlB/nJ+8WLFw9Uj0c96lGtyy5Y\nsKB12YMOOqh12fvuu6912UMOOaR12QceeKB1WY2OLWFJqsgQlqSKDGFJqsgQlqSKDGFJqsgQlqSK\nDGFJqsgQlqSKDGFJqsgQlqSKIjMHWyFiN+DvgUXAY4GXZOZXuh4/Adi/Z7UzMnPvCba3C3DRQJUQ\nALvvvnvrsh/60IdGUod77723ddlly5a1Lvvtb3+7ddlRXrY8yGXAL3zhCwfa9ihsuummrcvedddd\nI6yJGosy8+LJCkynJbwQ+CFwMDBRgp8OPAbYqrm9ahr7kaSxN/AEPpl5BnAGQETEBMXuy8xbZlIx\nSZoPRtUnvCQiboqIyyNieURsNqL9SNKcNoqpLE8HvgBcBTwR+Gfg6xGxaw7aAS1JY27oIZyZp3Td\nvSwiLgWuBJYA3xn2/iRpLhv5ELXMvAq4Fdhh1PuSpLlm5CEcEY8HNgduHPW+JGmuGbg7IiIWUlq1\nnZER20fEzsDtzW0ZpU94ZVPuA8DPgTOHUWFJGifT6RN+FqVvN5vbUc3yzwIHAU8HXgdsCtxACd93\nZ+b9M66tJI2Z6YwTPofJuzH+dPrVkaT5ZeDLlodeAS9b1iz2tKc9rXXZ888/v3XZRz7ykdOpzpS2\n2Wab1mWvu+66kdRBaxnJZcuSpCExhCWpIkNYkioyhCWpIkNYkioyhCWpIkNYkioyhCWpIkNYkioy\nhCWpIi9bloZkv/32a132pJNOGkkdVqxY0brs85///NZlb7vttulUR162LEmzmyEsSRUZwpJUkSEs\nSRUZwpJUkSEsSRUZwpJUkSEsSRUZwpJUkSEsSRUN/JP3kvo777zzaleBnXbaqXXZTTbZpHVZL1se\nHVvCklSRISxJFRnCklSRISxJFRnCklSRISxJFRnCklSRISxJFRnCklSRISxJFXnZsjQkixcvrl0F\nzUG2hCWpIkNYkioyhCWpIkNYkioyhCWpIkNYkioyhCWpIkNYkioyhCWpIkNYkioa6LLliDgE+DPg\nKcA9wPeApZn5864yGwBHA/sBGwBnAgdl5s3DqrS0ruy3336ty37qU58aYU00rgZtCe8GHAs8B3gB\nsD7wjYh4RFeZY4B9gJcBuwO/C3xh5lWVpPEzUEs4M/fuvh8RBwA3A4uAcyNiY+BA4JWZeU5T5vXA\nTyPi2Zl54VBqLUljYqZ9wpsCCdze3F9ECfazOgUy82fAtcCuM9yXJI2daYdwRASl6+HczPxJs3gr\n4DeZeWdP8ZuaxyRJXWYyn/By4KnAc1uUDUqLWZLUZVot4Yj4KLA3sCQzb+h6aCXw8KZvuNuWlNaw\nJKnLwCHcBPCLgT/OzGt7Hr4IeADYo6v8jsATgP+aQT0laSwNOk54OfAqYF9gVUQ8pnnojsy8NzPv\njIjjgaMj4lfAXcBHgPMcGSFJDzVon/AbKX27Z/csfz3wueb/bwNWA5+nXKxxBnDw9KsoSeNr0HHC\nU3ZfZOZ9wJubmyRpEv7a8jyx8ca950onduihh7Yue/rpp0+nOkP1jGc8o3XZV77ylQNt+ylPeUrr\nsgsXLhxo26Nw1FFHtS57ww03TF1II+cEPpJUkSEsSRUZwpJUkSEsSRUZwpJUkSEsSRUZwpJUkSEs\nSRUZwpJUkSEsSRVFZt251iNiF8oUmBqhxYsXty573nnnjbAmGtStt97auuz222/fuuyqVaumUx0N\nZlFmXjxZAVvCklSRISxJFRnCklSRISxJFRnCklSRISxJFRnCklSRISxJFRnCklSRISxJFflry9KQ\n3HXXXa3LnnLKKa3LLl++vHVZL0Wee2wJS1JFhrAkVWQIS1JFhrAkVWQIS1JFhrAkVWQIS1JFhrAk\nVWQIS1JFhrAkVWQIS1JFzh0xT1x77bWty37gAx9oXXbp0qXTqU41V1xxxUDljzvuuNZlv/a1r7Uu\ne/nllw9UD40vW8KSVJEhLEkVGcKSVJEhLEkVGcKSVJEhLEkVGcKSVJEhLEkVGcKSVJEhLEk1ZWbr\nG3AIcCFwJ3AT8CVgx54yZwNrum6rgeWTbHMXIL158+ZtDG+7TJWrg7aEdwOOBZ4DvABYH/hGRDyi\nq0wCnwQeA2wFPBZ4x4D7kaR5YaAJfDJz7+77EXEAcDOwCDi366G7M/OWGddOksbcTPuEN6W0fG/v\nWf6aiLglIi6NiMN7WsqSpMa0p7KMiACOAc7NzJ90PfRvwDXADcDTgSOBHYGXz6CekjSWZjKf8HLg\nqcAfdS/MzO4JWC+LiJXAtyJiu8y8agb7k6SxM63uiIj4KLA3sCQzb5yi+AVAADtMZ1+SNM4Gbgk3\nAfxi4HmZ2ebnGp5J6TeeKqwlad4ZKIQjYjnwKmBfYFVEPKZ56I7MvDcitgdeDXwduA3YGTgaOCcz\nVwyv2pI0HgZtCb+R0qo9u2f564HPAb+hjB9+K7AQ+CVwKvD+GdVSksbUoOOEJ+1DzszrgCUzqZAk\nzSfOHSFJFRnCklSRISxJFRnCklSRISxJFRnCklSRISxJFRnCklSRISxJFRnCklSRISxJFRnCklSR\nISxJFRnCklSRISxJFRnCklSRISxJFRnCklSRISxJFRnCklSRISxJFc2GEN6wdgUkaUSmzLfZEMLb\n1q6AJI3ItlMViMxcB/WYpAIRmwN7AlcD91atjCQNx4aUAD4zM2+brGD1EJak+Ww2dEdI0rxlCEtS\nRYawJFVkCEtSRYawJFVkCEtSRYawJFX0vxlhvMyhL3hZAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fe1e4acbb50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWEAAAFuCAYAAABQqakCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzt3XuYXFWZ7/HvK0qAZCAGBASVW4gzMoNKBEEB420y4gFE\nEAaZAUSPYhAdLwd1GATvRzlgFIiXQfGCooiiiAEUBRERURAUJTBcAnIJdwkGwiWs88fapZVKdfeu\n6qqsrurv53n6SXr3W3u/u6r617tWrb0rUkpIksp4UukGJGkyM4QlqSBDWJIKMoQlqSBDWJIKMoQl\nqSBDWJIKMoQlqSBDWJIKMoQHVEQcFBFPRMR2pXtpp+rtA13ednFEfKnXPdXcdtd9D5KIeGZEPBwR\nOzUt+3JE3NTl+p6IiM/0sL/NqnUe2Kt1jrCdGRHxl4iY28/tjGaoQjgipkbEByPinIi4d7QHMSJO\nqX7e+vXHNrVbRcQZEXFfRCyLiJ9HxJw2de3W1/g6r8t1vjUiDhphl7s+5zwiXhURR3d7+xoS3ff3\nxDhuO17j6XuQfAC4NKX0y6ZlQ7XvEbFeRHwhIu6qgvanEfH85pqU0n3AycBHynQJTy614T7ZADgK\nuBm4EpgzRv1y4I1ANC17oLkgIp4BXAo8BnwCeAh4A/CjiHhZSunipvJ/a7ON7YG3A38N4Q7XOQ+4\nG/jKGPvSqd2qdX+wx+ttWBt4vMvbPpscxCWMp++BEBEbAAcC/166l36JiAAWAv8EfBK4l/x8vzAi\ntksp3dBU/jng7RExJ6V04eruddhC+HZg45TSXRExG/j1GPWPp5ROG6Pm/cC6wDYppesBIuJkYBHw\nKXLIApBS+kbrjSPiZeSji292s84+irFLqsKINYAnpZQeq3ublNKjXXWVb1t7O702nr4HyL+TDwDO\nLt1IH70O2AnYO6V0JkBEfBu4jnzg8dcDppTSooi4GjgYuHB1NzpUwxEppcdSSnd1cpvIpo1SsjPw\n20ZYVtt5GDgL2C4iZo6y7jWB1wIXppRu73Sd1fjcNsCcpmGNn7ZsZkpEHN/0kuu7EbH+GPt8Cvmo\noHkIZUX1fWMs7l0R8Y6IuJ78iuEfIuIpEfGhiPhNRPy52t5FowzNfKDp+2OqZVtVY4/3V+v4UkSs\n1XLblcaEm8a/XzTWvlaP5zERcVs1zPOTiPiH1nWOct+M1PfWEXFq1fNdEfGh6ufPjIjvRcQDEXFH\nRLyrZX2d3GczIuJr1brujzxktm20GVaLiGdXw1n3Rh7b/XVE7D7W/lX2JA9FPFTj/nhPRPwiIu6J\niIeq/dh7lPrXR8SiqqffRMQubWo2qR73JRGxPCKujohDavTy5Gq/Nx5zD2FvYEkjgAFSSvcApwN7\nRsRTWurPB+refz01VCHchXWAB4Gl1ZP5xIiY2lIzBXi4zW0bT+DR3hh7NTAd+HqX63wHcCtwDXAA\n+a/3R5vqAziR/JLrGGAB+Yl04ig9QX759ePq/431tr40PQR4G/B54N3AfeSj90OAC4AjgKPJQ0Dn\nRsS2Y2yzMdZ4OjAVeB/wLeCgaj3taludwNj7+n/J452XAe8B/oc8FLT2GP2N1fe3qn/fSx5KOjIi\n/gP4Efkxem+1rWMjYuem29e6zyIiyEem+wGnAP8JPJ08DLXS/RER21Q9PBv4OPAu4C/A9yJiz9F2\nJvKrmu2B39bc/7cDV5CH+d5PPoI+PSJe1aZ2DvmV3Neq+hnAORHxnKbtbwj8CngZ8Jlq/f8DnBwR\nbx+jl03Jvwsfq9H386u+W11G/r2f1bL8N8D05l5Xm5TSUH4Bs8njigeO8POPkh/MfYB9gS9V9ReR\nX3o36r5PHk+a2nL7S4AVwDtH6eEMctiu27K89jqB3wM/bbPug6p+z21ZfhzwKPB3Y9w/JwAr2izf\nrFrv/cCMlp8F8OSWZesCdwD/3bL8CeADTd8fXS37Qkvdd4C7WpbdBHyp030FNqy+P6Ol7gPV7b/U\nur9t9n+kvhc0LXsScAt57PjdTcvXA5a19F7rPiO/YnoCeFtL7fnVc+LAlmW/bbPei4FFY+zfltV2\n5rX52SnAjS3LprR8vwbwO+DHbe63FcDzmpY9k3xgcUbTspPJf7Smt9z+G+Q/9FNanofN+71ZtY0v\n1ngcH2x9TlbLX1Wt45Uty3estrfPWOvu9dekPRJOKR2ZUvrPlNIZKaXTU0qHAEcCLyYHc8NngaeS\n//o/r3pZOp8c8jDCEVZE/B35za+zU0pLW37c1Trb7QbwhZZlPyf/omxWcx0jOSPld47/trHscfjr\ny/6nAmuSjyLqTJVL5CPr1n7Xj9GHhBq3HWtfX159/9mWuhNq9DbWtr/4129SeoK8z0EOrsbyB4Br\nyUHXWFb3PptL/gNycsu2T6Jp/L66/UuBbwPrRcT6jS/yUfnWEfH0UfalMXxzf439JqX0SNO2p5Of\ntz+n/eN9SUrpyqbb/ol8wPHP1ZE+5D82PwDWaNP7eiOst7G+m1NKa6SU3lij9bWBR9osX06+P1t/\nxxr3xwY11t1TkzaER/Ap8i/cKxoLUkrnkl+W7wJcTv4lexX55WKQXwa2sw952KF1KGI862znTy3f\nN55MT+1gHe0sbrewGp+9ivxkvhe4izzssl7N9d7S8n0n/Y61r40wvr65KKV0PzVDZxStfT8ALG/9\nQ1UtX2lfat5nmwF3pJSWt6zv+pbvZ5KfIx8mz5pp/jqmqtmwxv7UemM2Iv5XRPwyIh4mH6neBbyV\n9o93a6+Q3wibCmwQEU8jD8+9uU3vjfH6Or3X8TD596/VWuTf8dbhwMb9sdqn6A3b7IhxSSktj4h7\nyWNZzcsXRH4za1vy0cqVwJvID9h1I6zuAPIv5MIRttXNOttZMcLy2rMfRrDKmHVE/Bv5yO+75Gk/\nd1Xb/0+ajv7GMJ5+2902at52vNpte8x96dF91qxx4PT/aJr22KJdGDbcW/075h+96k2175NnDLyV\nPITyGHmMe/8avcLKj02j91MZecrl72qudyx3kMfUWzWW3d6yvHF/3NOj7ddmCDepXhJvQP7LvJKU\nZy/8qqn2leSg+kWb9WxMfpPiS2mUKU8119mvv8zdrHdv4IaUUvNwDY2ZAoU078fN1b8zm/5PRMxg\n/K8MulX3PruZPAtmrZaj4a1b6m6s/n0spdQ6U6aOW8jPsS1q1L62qp3bGFIBiIiRhgNae4X8BthD\n5HAL8ljtGl323okrybOQWu1Y9dN6oLMF+bl0TZ/7WsWkHI6IiCkjjEE2piadM8btXwTsBZycUnqw\nTcn+5CfcKkMRXaxzGfklXK8tq7a7bge3WcGq79S/kDwfcyL4CbnHeS3LDy/QS0Pd++w88ljx/26q\nC+Cw5tunlO4mH5m+pd1UrcgnYoyoCtPfAC/ooPe/HqxFxObkKW7t7BRNp9FHxDOBPYDzqrHxJ8hv\nxO5dzfDoqPcOp6idAWwUEa9tWf8+wFlp1bnos4EHUkqrnDHbb0N3JBwRh5FDa9Nq0R7VkwHgM1XA\nbQz8NiJOI58gAfAv5HHZhSmls5rW9yzytKqzgCXAPwJvIf+lPXKENg4Abk8p/WyEHjtZ5+XAoRFx\nJPll5l0ppQsaqxrpbhhheet6Azgh8inVK1JK3xrjNmcDr42I7wE/JL+cfgvwB2CsN9bGa8x9Tfkk\nnU8D74qI7wPnAs8lP7Z3U+aU3Lr32ffI06eOi4ityc/LPfjbH+Dm3g8jvzn2+4j4b/LR8UbkYN+U\nPD1rNN8HPhIR01JKo73/cDZ5+tt5EfGNahvzyFPK2k1JvJo8Je0E8hDbW6u+j2mqeR/5VeKvqt7/\nSB7+m02etjZaEDemqH2ZPCQymjOA/wBOqQL/nqr3NVr6aXgl+Q3D1W91T8fo9xd5etOKEb6eVdWs\nRx6Tupb88ugh8ljUEeSXSs3rm04ez7uN/NLsBvL0tqkjbH/ralufHKXH2uskv1FxFvDnar0/rZYf\nVH2/XUv9S6rlu45xPz0JmE/+I/A41XQ1/jYNqO3UO/J82Bur++w35D9cp5BfcjfXrQCOavr+6GpZ\n67S3xn48q2nZjTRNQ+pkX8mhfEx13/6F/K77LHIIn1Tj+VO371PIR06tt78AuKrL+2wGeY7tn8lv\ngp1MDtYngNe11G5ereM28ht+t5DD9TU19vFp5JkDr2+zT609HUz+g/AQ+Q/HgY37pM399mnyq8Br\nq/pfA7u02f4G5DnCi6veb6sep0OaahrPw66mqDX9nn+BPA7/IPmV0vPb1P19dR/P6SRrevUVVRPS\n0IqI9cizI45MKX28dD+diIjXkF/C75xWvtjOeNd7MjArpbRrr9Y5qKrpoTunlOoM0fTcpBwT1vCK\nllOgK+8kvyy+cPV205mImNLy/ZPI49lLaX/213h8EHhBNF3KcjKq3rRtnCNQxNCNCWvS2y8iDiaP\nvy4jz8X+V/LZdj07kuyTEyJiHeCX5Dmue5PfzX9/ajppohdSPpFinV6ucxClPM+7kzene84Q1rD5\nHXku6xHkX647ySfhHFWyqZouIL8R9mrySQXXk09jbj0DUENkQg5HRMRhEXFT5CsxXRoRq+PSjn0X\nEUdHjYvID4KI2CUizop8tbInImKPNjUfiojbI19968cxyhXneiWl9NuU0j+nlDZMKa2VUtospfTu\nVOOKYc3G2r9o/6EAbU/M6aD301JK26eUnppSWjul9E+9DuCIeH9EXBYRSyPizog4MyJmtdRMiYiT\nIl857cHIV2vr1ZlsfVVz/y5sedxWRMSCUj1PuBCOiP3IF2Y5mjzV5iryFJnVfk53n1xNnuqzcfXV\nbkL5IJhKnlK30jzWhoh4L/nU7LcAO5CHBs6LfHnPQTDq/lXOYeXHsu5ZZCXtQr6WxgvJp+c/hfxh\nAs3XUphPPhrfG9gV2IT85uAgqLN/jeuQNB67p5NfOZVRYkrGGNNKLgU+3fR9kK+6dETp3nqwb0cD\nV5Tuow/79QSwR8uy21n5anDrkqfj7Vu63x7t3ynAd0v31oN926Dav52bHqdHgL2aahqfdLJD6X7H\nu3/VsguA40v31viaUEfCkS+0PJs8nw/IV6EiX7pvWN7F3bp6iXtD5IuEP3PsmwyWiNiCfITR/Dgu\nJZ+iPSyPI+TTjO+MfBHzBdU77YNmOvnIsHEhotnk94qaH7tryfOQB/Gxa92/hgMi4u6I+H1EfKzl\nSHm1mmhvzG1APqPlzpbld5L/Gg+6S8mT368lvwQ6BrgoIv4xpbSsYF+9tjH5id/ucaxzyukgOIf8\nEv0mYCvyxdUXRsRO1YHDhFedFj0fuDj97XTdjYFH06qXXx24x26E/YN8OYGbya/WtiVfWGkWK1/C\ndrWZaCE8kmAIPgU2pdR81aurI+Iy8pNhX5quSzvEhuJxBEgpnd707R8i4vfkMx/nkF/uDoIFwHOo\n977EID52jf17cfPClFLzNZv/EBFLgPMjYouU0k2rs0GYeG/M3UM+LXGjluUbsupR1cBL+SLg15Gv\n+jVMlpB/aSfF4whQ/fLew4A8lhFxIvlDB+aklT//cAmwZqx6YaeBeuxa9u+OMcp/RX6+FnnsJlQI\np3xlo8vJn5AA/PUlxcvJH/0zVCJfyW0r8rVPh0YVSEtY+XFcl/yO9dA9jgAR8Qzyp1ZM+MeyCqg9\ngZemlFovVn85+VoizY/dLOBZ5JNIJrwx9q+d55OP8os8dhNxOOJ44CsRcTn5qlLvJJ/Z8+WSTfVC\nRBxLvlLTzeQrQn2Q/IQ/rWRf3Yj8gaiNT3kA2DIingvcl/LZWPOB/4r8ac2LyZ8EcSv5IjMT3mj7\nV30dTR4TXlLVfYL8qmakC61PCNV82P3JV2hbFhGNVysPpJSWp5SWRsQXgeMj4n7yhW8+A/wipXRZ\nma7rG2v/ImJL4PXkD1u4l3yVveOBn6WUri7Rc/HpGSNMK5lH/sV9mPzX9wWle+rRfp1GDqKHye82\nfwPYonRfXe7LS/jbhzs2fzV/yOUx5Dc/HiKH08zSffdi/8hns51LDuDl5CukfRZ4Wum+a+xXu31q\nvVrZFPJc23vIIfxtYMPSvfdi/4BnkK8hcnf1vLyW/KbqtFI9exU1SSpoQo0JS9JkYwhLUkGGsCQV\nZAhLUkGGsCQVZAhLUkHFT9aIiPWBufztk1cladCtRf5E7PNSSveOVti3EI6Iw4D3kK+8dBVweErp\n121K55KvaiRJw+YA8klZI+rLcESHn46xuB89SNIEsHisgn6NCb8T+HxK6asppUXAoeRTBA9pU+sQ\nhKRhNWa+9TyEJ8mnY0hST/TjSHi0T8cYqCvzS1K/rc4paoN4ZX5J6qt+hPCk+nQMSRqPnodwmmSf\njiFJ49GvecJD++kYktRLfQnhlNLp1ZzgD5GHJa4E5qaU7u7H9iRpUBX/ZI2I2I48fCFJw2Z2SumK\n0Qq8gI8kFWQIS1JBhrAkFWQIS1JBhrAkFWQIS1JBhrAkFWQIS1JBhrAkFWQIS1JBhrAkFWQIS1JB\nhrAkFWQIS1JBhrAkFWQIS1JBhrAkFWQIS1JBhrAkFWQIS1JBhrAkFWQIS1JBhrAkFWQIS1JBhrAk\nFWQIS1JBhrAkFWQIS1JBhrAkFWQIS1JBhrAkFWQIS1JBTy7dgDSRzZ07t3btoYceWrt2zz337Kad\nMUVE7dqjjjqqdu03v/nN2rXXX3997Vp5JCxJRRnCklSQISxJBRnCklSQISxJBRnCklSQISxJBRnC\nklSQISxJBRnCklSQISxJBUVKqbcrjDgaOLpl8aKU0nNGqN8OuLynTUgj2HXXXTuqP/PMM2vXTp8+\nvdN2BsaiRYtq126zzTZ97GTgzE4pXTFaQb8u4HM18HKgcTWRx/u0HUkaaP0K4cdTSnf3ad2SNDT6\nNSa8dUTcFhE3RMSpEfHMPm1HkgZaP0L4UuBgYC5wKLAFcFFETO3DtiRpoPV8OCKldF7Tt1dHxGXA\nzcC+wCm93p4kDbK+T1FLKT0AXAfM7Pe2JGnQ9D2EI2IasBVwR7+3JUmDpuchHBHHRsSuEbFZRLwI\nOJM8Re20Xm9LkgZdP6aoPQP4BrA+cDdwMbBjSunePmxLkgZaP96Y27/X65SkYeVH3mtS6fTU4mE+\nFbkTm2++eekWhpYX8JGkggxhSSrIEJakggxhSSrIEJakggxhSSrIEJakggxhSSrIEJakggxhSSrI\n05Y1IU2bNq127V577VW79vjjj++mnUlvrbXWql173HHH1a5997vf3U07Q8UjYUkqyBCWpIIMYUkq\nyBCWpIIMYUkqyBCWpIIMYUkqyBCWpIIMYUkqyBCWpII8bVmrzcKFC2vXzpgxo3bt9ttv30076pN9\n9923du13vvOdjtZ9ySWXdNrOhOeRsCQVZAhLUkGGsCQVZAhLUkGGsCQVZAhLUkGGsCQVZAhLUkGG\nsCQVZAhLUkGetqxx2X333WvXvvSlL61du+aaa3bTjiaATTbZpHZtJ5/MDLDTTjt12s6E55GwJBVk\nCEtSQYawJBVkCEtSQYawJBVkCEtSQYawJBVkCEtSQYawJBVkCEtSQZ62rFV0cnrxm970ptq1g3gq\n8vLly2vX/uAHP6hde80119Su/dSnPlW7du7cubVrTz755Nq106ZNq12rznR8JBwRu0TEWRFxW0Q8\nERF7tKn5UETcHhEPRcSPI2Jmb9qVpOHSzXDEVOBK4DAgtf4wIt4LvA14C7ADsAw4LyIG7zBIkvqs\n4+GIlNK5wLkAERFtSt4BfDil9IOq5kDgTuA1wOndtypJw6enb8xFxBbAxsBPGstSSkuBXwHDdw06\nSRqnXs+O2Jg8RHFny/I7q59JkpqsrilqQZvxY0ma7HodwkvIgbtRy/INWfXoWJImvZ6GcErpJnIQ\nv7yxLCLWBV4IXNLLbUnSMOh4dkRETAVmko94AbaMiOcC96WU/gTMB/4rIq4HFgMfBm4Fvt+TjiVp\niHRzxtwLgAvIY7wJaHxS31eAQ1JKn4yIdYDPA9OBnwOvSik92oN+JWmoREpl3y+LiO2Ay4s2MQns\nt99+tWsXLFhQu3b69OndtFPMJZd0Nip25JFH1q696KKLOm2nqEWLFtWu3XrrrfvSw2WXXdZR/QB+\n2vLslNIVoxV4AR9JKsgQlqSCDGFJKsgQlqSCDGFJKsgQlqSCDGFJKsgQlqSCDGFJKsgQlqSC/LTl\nAbb77rvXrv3a175Wu3aNNdbopp1izj///Nq1nXzCMAzeqcg777xz7dr111+/j52oLo+EJakgQ1iS\nCjKEJakgQ1iSCjKEJakgQ1iSCjKEJakgQ1iSCjKEJakgQ1iSCvK05QlmvfXWq1375je/uXbtoJ2K\n/MMf/rB27f7771+7dtmyZd20MzB23HHH2rUzZszoYyf1XHXVVaVbKM4jYUkqyBCWpIIMYUkqyBCW\npIIMYUkqyBCWpIIMYUkqyBCWpIIMYUkqyBCWpIIMYUkqyGtHrAbrrLNO7dr58+fXrt1tt926aaeY\nhQsX1q594xvfWLt2EK8HMWXKlNq173vf+2rX7rPPPt2001Nnn3127dojjjiij50MBo+EJakgQ1iS\nCjKEJakgQ1iSCjKEJakgQ1iSCjKEJakgQ1iSCjKEJakgQ1iSCur4tOWI2AX4P8Bs4OnAa1JKZzX9\n/BTgoJabnZtSGqxzbHvogAMOqF174IEH9rGT3jv//PNr177uda+rXbt8+fJu2hkYn/jEJ2rXHn74\n4X3spPcWL15cu3bp0qX9a2RAdHMkPBW4EjgMSCPUnANsBGxcfe3fVXeSNOQ6PhJOKZ0LnAsQETFC\n2SMppbvH05gkTQb9GhOeExF3RsSiiFgQETP6tB1JGmj9uJTlOcB3gJuArYCPAwsjYqeU0kjDF5I0\nKfU8hFNKpzd9+4eI+D1wAzAHuKDX25OkQdb3KWoppZuAe4CZ/d6WJA2avodwRDwDWB+4o9/bkqRB\n08084anko9rGzIgtI+K5wH3V19HkMeElVd0ngOuA83rRsCQNk27GhF9AHttN1ddx1fKvAPOAbYED\ngenA7eTw/UBK6bFxdytJQ6abecI/Y/RhjH/pvh1Jmlz8tGWt4sYbb6xdO+ynIj/vec+rXXvwwQfX\nrp03b14X3ZTzyCOP1K495phj+tfIEPICPpJUkCEsSQUZwpJUkCEsSQUZwpJUkCEsSQUZwpJUkCEs\nSQUZwpJUkCEsSQV52rJWcdJJJ9WunQiflrv22mvXrt1hhx06Wvepp55au3aTTTbpaN2D5HOf+1zt\n2vvvv7+PnQwfj4QlqSBDWJIKMoQlqSBDWJIKMoQlqSBDWJIKMoQlqSBDWJIKMoQlqSBDWJIK8rRl\nrWKrrbaqXfue97ynj53UM3fu3Nq1L3vZy/rYyWC59dZba9d2cvq2OuORsCQVZAhLUkGGsCQVZAhL\nUkGGsCQVZAhLUkGGsCQVZAhLUkGGsCQVZAhLUkGetqxVzJs3r3QL6lInpyKfeeaZtWuvuOKKbtpR\nDR4JS1JBhrAkFWQIS1JBhrAkFWQIS1JBhrAkFWQIS1JBhrAkFWQIS1JBhrAkFdTRacsR8X5gL+Dv\ngYeBS4D3ppSua6qZAhwP7AdMAc4D5qWU7upV04PmggsuqF173XXXjV1UmTVrVjftaMA88MADtWv3\n2muv2rWeijwxdHokvAtwAvBC4BXAU4AfRcTaTTXzgVcDewO7ApsA3xl/q5I0fDo6Ek4p7db8fUQc\nDNwFzAYujoh1gUOAf00p/ayqeQNwTUTskFK6rCddS9KQGO+Y8HQgAfdV388mB/tPGgUppWuBW4Cd\nxrktSRo6XYdwRAR56OHilNIfq8UbA4+mlJa2lN9Z/UyS1GQ81xNeADwH2LlGbZCPmCVJTbo6Eo6I\nE4HdgDkppdubfrQEWLMaG262IfloWJLUpOMQrgJ4T+ClKaVbWn58OfA48PKm+lnAs4BfjqNPSRpK\nnc4TXgDsD+wBLIuIjaofPZBSWp5SWhoRXwSOj4j7gQeBzwC/cGaEJK2q0zHhQ8ljuxe2LH8D8NXq\n/+8EVgBnkE/WOBc4rPsWJWl4dTpPeMzhi5TSI8Dh1ZckaRSRUtlJCxGxHXksWcCmm25au/bss8+u\nXbvtttt204765JZbWt9OGVknpyJfeeWV3bSj/pmdUhr1/HAv4CNJBRnCklSQISxJBRnCklSQISxJ\nBRnCklSQISxJBRnCklSQISxJBRnCklTQeC7qrj647bbbatd+/etfr107bdq02rVbbrll7dph99hj\nj9Wu/chHPlK79tRTT61du3jx4tq1GjweCUtSQYawJBVkCEtSQYawJBVkCEtSQYawJBVkCEtSQYaw\nJBVkCEtSQYawJBXkpy1PEjNnzqxd+5KXvKQvPcyfP7927TrrrNOXHq666qqO6o899tjataeddlqn\n7Wj4+WnLkjSRGcKSVJAhLEkFGcKSVJAhLEkFGcKSVJAhLEkFGcKSVJAhLEkFGcKSVJAhLEkFee0I\nSeofrx0hSROZISxJBRnCklSQISxJBRnCklSQISxJBRnCklSQISxJBRnCklSQISxJBXUUwhHx/oi4\nLCKWRsSdEXFmRMxqqbkwIp5o+loREQt627YkDYdOj4R3AU4AXgi8AngK8KOIWLupJgFfADYCNgae\nDhwx/lYlafg8uZPilNJuzd9HxMHAXcBs4OKmHz2UUrp73N1J0pAb75jwdPKR730tyw+IiLsj4vcR\n8bGWI2VJUqWjI+FmERHAfODilNIfm370deBm4HZgW+CTwCxgn3H0KUlDqesQBhYAzwFe3LwwpXRy\n07d/iIglwPkRsUVK6aZxbE+Shk5XwxERcSKwGzAnpXTHGOW/AgKY2c22JGmYdXwkXAXwnsBLUkq3\n1LjJ88njxmOFtSRNOh2FcDXfd39gD2BZRGxU/eiBlNLyiNgSeD2wELgXeC5wPPCzlNLVvWtbkoZD\np0fCh5KPai9sWf4G4KvAo+T5w+8ApgJ/Ar4NfHRcXUrSkOp0nvCoY8gppVuBOeNpSJImE68dIUkF\nGcKSVJAhLEkFGcKSVJAhLEkFGcKSVJAhLEkFGcKSVJAhLEkFGcKSVJAhLEkFGcKSVJAhLEkFGcKS\nVJAhLEndCPfMAAAAg0lEQVQFGcKSVJAhLEkFGcKSVJAhLEkFGcKSVJAhLEkFTYQQXqt0A5LUJ2Pm\n20QI4c1LNyBJfbL5WAWRUloNfYzSQMT6wFxgMbC8aDOS1BtrkQP4vJTSvaMVFg9hSZrMJsJwhCRN\nWoawJBVkCEtSQYawJBVkCEtSQYawJBVkCEtSQf8f4H9dAetxoh8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fe1e49c0bd0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "nsample = 3\n",
    "rand_idx = np.random.randint(mnist.train.images.shape[0], size=nsample)\n",
    "\n",
    "for i in rand_idx:\n",
    "    curr_img = np.reshape(mnist.train.images[i, :], (28,28))\n",
    "    curr_lbl = np.argmax(mnist.train.labels[i, :])\n",
    "    plt.matshow(curr_img, cmap=plt.get_cmap('gray'))\n",
    "    plt.title(\"\"+str(i)+\"th training image \"\n",
    "              + \"(label: \" + str(curr_lbl) + \")\")\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 이미지 분류를 위한 RNN 구성\n",
    "\n",
    "28x28 이미지를 RNN 입력을 위해 아래와 같이 구성한다.\n",
    "\n",
    "* input_vec_size: 하나의 이미지 (28x28) 중 한 행씩 RNN의 입력으로 사용한다.\n",
    "* time_step_size: 하나의 이미지 (28x28)를 모두 입력하려면 28개의 행이 필요하다. 따라서 time_step은 28.\n",
    "* lstm_size: rnn_cell에 포함된 hidden unit의 수"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# configuration\n",
    "#                        O * W + b -> 10 labels for each image, O[? 28], W[28 10], B[10]\n",
    "#                       ^ (O: output 28 vec from 28 vec input)\n",
    "#                       |\n",
    "#      +-+  +-+       +--+\n",
    "#      |1|->|2|-> ... |28| time_step_size = 28\n",
    "#      +-+  +-+       +--+\n",
    "#       ^    ^    ...  ^\n",
    "#       |    |         |\n",
    "# img1:[28] [28]  ... [28]\n",
    "# img2:[28] [28]  ... [28]\n",
    "# img3:[28] [28]  ... [28]\n",
    "# ...\n",
    "# img128 or img256 (batch_size or test_size 256)\n",
    "#      each input size = input_vec_size=lstm_size=28"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "input_vec_size = lstm_size = 28\n",
    "time_step_size = 28\n",
    "\n",
    "batch_size = 128\n",
    "test_size = 256"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## RNN 모델 정의"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def init_weights(shape):\n",
    "    return tf.Variable(tf.random_normal(shape, stddev=0.01))\n",
    "\n",
    "def model(X, W, B, lstm_size):\n",
    "    # X, input shape: (batch_size, time_step_size, input_vec_size)\n",
    "    XT = tf.transpose(X, [1, 0, 2])  # permute time_step_size and batch_size\n",
    "    # XT shape: (time_step_size, batch_size, input_vec_size)\n",
    "    XR = tf.reshape(XT, [-1, lstm_size]) # each row has input for each lstm cell (lstm_size=input_vec_size)\n",
    "    # XR shape: (time_step_size * batch_size, input_vec_size)\n",
    "    X_split = tf.split(0, time_step_size, XR) # split them to time_step_size (28 arrays)\n",
    "    # Each array shape: (batch_size, input_vec_size)\n",
    "\n",
    "    # Make lstm with lstm_size (each input vector size)\n",
    "    lstm = tf.nn.rnn_cell.BasicLSTMCell(lstm_size, forget_bias=1.0, state_is_tuple=True)\n",
    "\n",
    "    # Get lstm cell output, time_step_size (28) arrays with lstm_size output: (batch_size, lstm_size)\n",
    "    outputs, _states = tf.nn.rnn(lstm, X_split, dtype=tf.float32)\n",
    "\n",
    "    # Linear activation\n",
    "    # Get the last output\n",
    "    return tf.matmul(outputs[-1], W) + B, lstm.state_size # State size to initialize the stat"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 데이터 변환\n",
    "\n",
    "* training과 test 데이터를 1d에서 2d형태로 reshape한다.\n",
    "* 입력과 출력에 해당하는 placeholder 선언"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "trX, trY, teX, teY = mnist.train.images, mnist.train.labels, mnist.test.images, mnist.test.labels\n",
    "trX = trX.reshape(-1, 28, 28)\n",
    "teX = teX.reshape(-1, 28, 28)\n",
    "\n",
    "X = tf.placeholder(\"float\", [None, 28, 28])\n",
    "Y = tf.placeholder(\"float\", [None, 10])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 모델 선언\n",
    "\n",
    "* Hidden unit과 output unit크기만큼 모델 생성\n",
    "* 모델 생성"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# get lstm_size and output 10 labels\n",
    "W = init_weights([lstm_size, 10])\n",
    "B = init_weights([10])\n",
    "\n",
    "py_x, state_size = model(X, W, B, lstm_size)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## loss 함수 선언 및 최적화"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(py_x, Y))\n",
    "train_op = tf.train.RMSPropOptimizer(0.001, 0.9).minimize(cost)\n",
    "predict_op = tf.argmax(py_x, 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 세션 생성 및 학습 정확도 측정"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(0, 0.68359375)\n",
      "(1, 0.859375)\n",
      "(2, 0.90234375)\n",
      "(3, 0.9296875)\n",
      "(4, 0.95703125)\n",
      "(5, 0.90234375)\n",
      "(6, 0.9453125)\n",
      "(7, 0.94140625)\n",
      "(8, 0.94921875)\n",
      "(9, 0.98046875)\n",
      "(10, 0.96875)\n",
      "(11, 0.96875)\n",
      "(12, 0.984375)\n",
      "(13, 0.96484375)\n",
      "(14, 0.98828125)\n",
      "(15, 0.98046875)\n",
      "(16, 0.953125)\n",
      "(17, 0.98046875)\n",
      "(18, 0.98828125)\n",
      "(19, 0.96875)\n",
      "(20, 0.96875)\n",
      "(21, 0.95703125)\n",
      "(22, 0.96875)\n",
      "(23, 0.96484375)\n",
      "(24, 0.98828125)\n",
      "(25, 0.98046875)\n",
      "(26, 0.98046875)\n",
      "(27, 0.97265625)\n",
      "(28, 0.96875)\n",
      "(29, 0.98828125)\n",
      "(30, 0.9765625)\n",
      "(31, 0.97265625)\n",
      "(32, 0.9609375)\n",
      "(33, 0.9609375)\n",
      "(34, 0.96484375)\n",
      "(35, 0.98046875)\n",
      "(36, 0.98046875)\n",
      "(37, 0.984375)\n",
      "(38, 0.98046875)\n",
      "(39, 0.9765625)\n",
      "(40, 0.98046875)\n",
      "(41, 0.9765625)\n",
      "(42, 0.9921875)\n",
      "(43, 0.97265625)\n",
      "(44, 0.96875)\n",
      "(45, 0.9765625)\n",
      "(46, 0.984375)\n",
      "(47, 0.98828125)\n",
      "(48, 0.95703125)\n",
      "(49, 0.9921875)\n",
      "(50, 0.96875)\n",
      "(51, 0.98046875)\n",
      "(52, 0.984375)\n",
      "(53, 0.9765625)\n",
      "(54, 0.96875)\n",
      "(55, 0.97265625)\n",
      "(56, 0.953125)\n",
      "(57, 0.97265625)\n",
      "(58, 0.984375)\n",
      "(59, 0.984375)\n",
      "(60, 0.9765625)\n",
      "(61, 0.97265625)\n",
      "(62, 0.9765625)\n",
      "(63, 0.984375)\n",
      "(64, 0.95703125)\n",
      "(65, 0.98046875)\n",
      "(66, 0.98828125)\n",
      "(67, 0.9765625)\n",
      "(68, 0.9765625)\n",
      "(69, 0.9765625)\n",
      "(70, 0.97265625)\n",
      "(71, 0.953125)\n",
      "(72, 0.9921875)\n",
      "(73, 0.96484375)\n",
      "(74, 0.9765625)\n",
      "(75, 0.96484375)\n",
      "(76, 0.97265625)\n",
      "(77, 0.95703125)\n",
      "(78, 0.97265625)\n",
      "(79, 0.9765625)\n",
      "(80, 0.9609375)\n",
      "(81, 0.96875)\n",
      "(82, 0.96875)\n",
      "(83, 0.95703125)\n",
      "(84, 0.984375)\n",
      "(85, 0.98828125)\n",
      "(86, 0.98828125)\n",
      "(87, 0.97265625)\n",
      "(88, 0.9765625)\n",
      "(89, 0.984375)\n",
      "(90, 0.98046875)\n",
      "(91, 0.96875)\n",
      "(92, 0.98828125)\n",
      "(93, 0.97265625)\n",
      "(94, 0.97265625)\n",
      "(95, 0.95703125)\n",
      "(96, 0.98828125)\n",
      "(97, 0.96875)\n",
      "(98, 0.98046875)\n",
      "(99, 0.97265625)\n"
     ]
    }
   ],
   "source": [
    "# Launch the graph in a session\n",
    "with tf.Session() as sess:\n",
    "    # you need to initialize all variables\n",
    "    tf.initialize_all_variables().run()\n",
    "\n",
    "    for i in range(100):\n",
    "        for start, end in zip(range(0, len(trX), batch_size), range(batch_size, len(trX)+1, batch_size)):\n",
    "            sess.run(train_op, feed_dict={X: trX[start:end], Y: trY[start:end]})\n",
    "\n",
    "        test_indices = np.arange(len(teX))  # Get A Test Batch\n",
    "        np.random.shuffle(test_indices)\n",
    "        test_indices = test_indices[0:test_size]\n",
    "\n",
    "        print(i, np.mean(np.argmax(teY[test_indices], axis=1) ==\n",
    "                         sess.run(predict_op, feed_dict={X: teX[test_indices],\n",
    "                                                         Y: teY[test_indices]})))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
